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ABSTRACT 

We present an updated version of McgaZ-LRG (jCoUister et al.ll2007D with photo- 
metric redshifts derived with the neural network method, ANNz as well as five other 
publicly available photo-z codes (HyperZ, SDSS, Le PHARE, BPZ and ZEBRA) for 
~1.5 million Luminous Red Galaxies (LRGs) in SDSS DR6. This allows us to identify 
how reliable codes are relative to each other if used as described in their public release. 
We compare and contrast the relative merits of each code using ~ 13000 spectroscopic 
redshifts from the 2SLAQ sample. We find that the performance of each code depends 
on the figure of merit used to assess it. As expected, the availability of a complete 
training set means that the training method performs best in the intermediate red- 
shift bins where there are plenty of training objects. Codes such as Le PHARE, which 
use new observed templates perform best in the lower redshift bins. All codes pro- 
duce reasonable photometric redshifts, the 1-a scatters ranging from 0.057 to 0.097 
if averaged over the entire redshift range. We also perform tests to check whether a 
training set from a small region of the sky such as 2SLAQ produces biases if used 
to train over a larger area of the sky. We conclude that this is not likely to be a 
problem for future wide-field surveys. The complete photometric redshift catalogue 
including redshift estimates and errors on these from all six methods can be found at 
www. star. ucl.ac.uk/^mbanerji/MegaZLRGDR6/megaz. html 
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1 INTRODUCTION 

Photometric redshifts will be one of the key ingredients for 
us to improve our understanding of the Universe in the 
decade to come. Up to date, galaxy large scale structure sur- 
veys relied mainly on spectroscopic redshifts to produce high 
precision powe r spectrum measurements of the gala xy dis- 
tribution fe.g. ICole et al.ll2005l : IPercival et al.ll2007l ). Com- 
bined with CMB experiments these surveys have provided 
evidence that the Universe is flat a nd is likely to be dom i- 
nated by a dark energy component (jKomatsu et al.ll2008l ). 

However having a considerable step up in the size of 
the spectroscopic surveys will be a hard task to achieve for 
technical reasons. Several Multi-fibre optical spectrographs 
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are currently being buih (FMOS) (jPahon et al.ll2006t ) or be- 
ing designed (WFMOS), but it is unlikely that they will be 
able to survey a considerable part of the sky. On the other 
hand radio interferometers may be able to p erform spec- 
troscopic surveys of the sky reaso nably quickly (|Blake et al.l 
12004 lAbdalla fc Rawlingj [20051) but the timescale for the 
technical advances to allow for this will be relatively long. 

The alternative to a full spectroscopic survey is to ob- 
tain multi-colour images of the sky and perform photo- 
metr ic redshift estimate s for the galaxies we have available 
(e.g. ICsabai et al]|2003l l. In a pilot study with high red- 
shift Luminous Red Galaxies (LRGs) it ha s been shown 
(|Padmanabhan et all [20061 : iBlake et al1l2007h that produc- 
ing large scale measurements with photometric redshifts is 
possible and competitive with a s maller spectr o scopi c red- 
shift survey. Using the same dataset [Blake et al.l (|2008l ) have 
shown that photometric redshifts can also be used to study 
small scale halo model signatures. 
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Figure 1. Map of the MegaZ-LRG sample (blue) covering tlie SDSS DR,6 area as well as the 2SLAQ sample (red). For clarity only a 
random subsample of galaxies have been plotted. 



On the other hand there are many caveats of photomet- 
ric redshifts that have to be assessed in order for us to be 
completely confident that these measurements are reliable 
to the level o f systematic s that we expect in future surveys. 
For instance iBlake et all (|2007l ) have performed a detailed 
study of whether star-galaxy separation influences the cos- 
mological measurements given that the LRGs that have been 
selected are contaminated at the per cent level by M-type 
stars which have similar colours. They have also assessed 
whether there is a significant contamination from dust cor- 
rections in the galaxy, by obtaining estimates of the power 
spectrum in different regions of obscuration in the sky. 

We extend this analysis concentrating on the level of 
systematic efl'ects that is introduced by the use of differ- 
ent photometric redshift techniques. We have selected the 
same sample as was selected in the MegaZ-LRG catalogue 
l|Collister et al.|[2007h and used several different photometric 
redshift techniques on the same galaxies available from the 
literature, including artificial neural networks, template fit- 
ting techniques and Bayesian techniques. We note here that 
LRGs have well defined 4000 A break, hence this strong fea- 
ture makes photometric redshift estimation an easier task. 
Here all codes compared produce reasonable photometric 
redshifts and we are comparing more subtle differences be- 
tween codes. 



In Sec[2]we describe the MegaZ-LRG data used. In Sec[3] 
we describe all the methods we have used to estimate the 
photometric redshifts for the LRG sample. In Sec|3]we com- 
pare different statistics for the different photo-z results. We 
perform an analysis to check for gradients across the sky 
which could arise from training sets if they only belong to a 
small area of the sky in Sec[5] and we present the catalogue 
in SecO Our conclusions are drawn in Seel?] 



2 DATA 

We use galaxy photometry in a DR6 equivalent to the 
MegaZ-LRG catalogue, a photometric-redshift catalogue of 
Luminous Red Galaxies based on the imaging component of 
the SDSS 4th Data Release. The constructi on of this cat- 
alogue follows the same prescription as in ICollister et al.l 
(,200^ ). Here we only outline briefly the description of the 
cat alogue. For details on the construction of this catalogue 
see lCoUister et all |20o3). 



2.1 Selection criteria 

The MegaZ-LRG catalogue is selected from the SDSS imag- 
ing database using a series of colour and magnitude cuts 
(|CoUister et al.ll2007D which were designed to match the se- 
lection c riteria of the 2dF-SD SS LRG and Quasar (2SLAQ) 
survey (| Cannon et all l2006l ). 2SLAQ is a spectroscopic 
follow-up combining the SDSS photometric survey and the 
spectroscopy from the Two-degree Field (2dF) instrument 
of the Anglo- Australian Telescope (AAT). 

The spectroscopic redshifts available from 2SLAQ were 
used to train and test the photometric redshift code, which 
we then applied to the entire set of LRGs selected from the 
SDSS imaging database. Around 13,000 objects in selected 
fields of the SDSS equatorial stripe (at declination (5 ~ 0°) 
were available. The 2SLAQ survey demonstrated that these 
selection criteria are ~ 95% efficient in the identification of 
intermediate-redshift LRGs. The most significant contami- 
nant, accounting for virtually all of the remaining ~ 5% of 
objects, is M-type stars. 

The 2SLAQ selection criteria fiuctuated a little at the 
beginning of the survey. Specifically, the faint limit of the 
i-band magnitude idcv, and the minimum value of dx (a 
colour variable used to select LRGs), were varied slightly. For 
the majority of the 2SLAQ survey, the criteria idev ^ 19.8 
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Code 


Authors 


Method 


Web link 


MyperZ 


Bozonella et al. 


Template 


u n — u u ■ — clZ 71 

http: / / webast.ast.obs-mip.ir/hyperz/ 1 


SDSS template 


SDSS pipeline 


LRG Template 


N/A code obtained from N. Padmanabhan 


BPZ 


Bcnitez 


Template + Bayesian priors 


http: / / acs.pha.jhu.edu/ ~txitxo/bpzdoc.html| 


ANNz 


CoUister & Lahav 


Neural Networks 


http: / /zuscrver2. star. ucl.ac.uk/ ~lahav/annz.html| 


ZEBRA 


Feldmann et al. 


Template, Bayesian, Hybrid 


www. cxp- astro . phys . ethz . ch/ Z EB R A 


Kcorrcct 


Blanton 


Model templates 


|http://cosmo. nyu.edu/blanton/kcorrcct/ 


Le PHARE 


Arnouts & Ilbert 


Template 


www.oamp.fr/people/arnouts/LE_PHARE.html 


EAZY 


Brammer at al. 


Template 


www.astro.yale.edu/eazy/ 


LRT Libraries 


Assef ot al. 


Template 


|http://www.astronomy.ohio-state.edu/~rjassef/lrt/| 



Table 1. Publicly available software packages for photo-z estimation, to date and to our knowledge. In this work we have used six 
representative codes from this tabic, namely HypcrZ, SDSS, BPZ, ANNz, ZEBRA and Le PHARE. 



and d± ^ 0.55 wer e used. For further details on this see 
l|Cannon et al.ll2006l '). 

We note that our training sub-sample is extrapolated 
in sky position. The 2SLAQ targets lie exclusively in the 
equatorial stripe at declination 5 « 0°, so may not fully 
trace effects such as dust extinction which depend on sky co- 
ordinate. One of the important aims of this study is to assess 
how much this sky extrapolation biases the final photo-z 
measurements with a training set method. 



3 PHOTOMETRIC REDSHIFTS ESTIMATORS 

This section describes how we obtained the photometric red- 
shifts for 5482 galaxies in the 2SLAQ sample. We have sub- 
divided the galaxies from 2SLAQ into a training sample and 
a testing sample. The training sample was used to train the 
training part of the codes presented here. The rest of the 
sample was chosen to be 5482 so that enough galaxies were 
left in the training sample. These galaxies were randomly 
chosen. Only these galaxies were used to test the codes fi- 
nal accuracy, hence there being less galaxies than the 13000 
galaxies available to test the codes. 

We have used several different codes in the literature 
to provide photometric redshift estimates as well as redshift 
errors for a subsection of the 2SLAQ sample. The rest of the 
sample was used by some of the methods as a training set 
and therefore we do not use those galaxies in the comparison 
as this might introduce biases in our study. 

We emphasise here that the comparison we are un- 
dertaking is a high level comparison; i.e. we are compar- 
ing end products without decomposing the problem into 
smaller parts in order to potentially assess where discrep- 
ancies are arising, in other words comparing codes as black 
boxes. Therefore the comparison is a comparison of the en- 
semble of codes plus galaxy libraries used with each code. We 
argue that this is a valid comparison as this is what a naive 
user of these publicly available codes would get should they 
choose one code rather than another. We also argue that 
a full analysis is needed to have the highest level of confi- 
dence in photometric redshifts and believe that this is being 
done by the Photometric accuracy testing program (PHAT) 
coUaboratiorQ. 



^ http:/ /www. astro. caltech.edu/twiki_phat/bin/vicw/ 



3.1 Methodology: codes considered in this work 

We give here a brief description of the codes we have used in 
this work. For a more ge neral descript i on of phot o-z meth- 
ods w e refer the reader to lLahav et al.l (|2008l ) and lBudavaril 
(2QQi). 

3.1.1 SDSS template fitting eode. 

The template-fitting technique in photometric redshift esti- 
mation is a fit between the data and a given set of tem- 
plates for those galaxies. For the purpose of redshift estima- 
tion, the galaxy templates usually come from stellar popula- 
tion synthesis models (e .g. iFioc &i Rocca-Vol mcranac 1993; 
iBruzual fc Charlotll2003l ) . A linear combination of templates 
is used. The coefficients d of the templates are the free pa- 
rameters for the minimisation. We note ^''(2) the set of tem- 
plates observed at redshift z and fa the observed flux in filter 
a with error of aa . The photometric redshift is found via 

x{c.,^)=j:[ j (1) 

where Rc,{^) is the fiux of spectrum ^l/ through filter a. 

The greatest disadvantage of this method (which also 
applies to all the other template fitting techniques presented 
here) is the potential mismatch between the templates used 
for the fitting and the properties of the sample of galaxies for 
which one wants to estimate the redshifts. A hybrid method 
can be used, in which in order to calibrate the templates 
to a better representation of the studied galaxy sample one 
would use a training set with known spectroscopic redshifts 
and similar properties to the galaxies whose redshifts need 
to be estimated. The SDSS code used here applies a hybrid 
method to the LRG sample using a modified elliptical galaxy 
template, adjusted to represent an LRG spectrum after three 
iterations of correction. Given that early type galaxies evolve 
passively, only one template is used in the code. 

3.1.2 Hyper Z 

HyperZ (jBolzonella et aLlbOOGD was the first publicly avail- 
able photo-z code and has consequently been widely used 
in the literature for photometric redshift estimation. It is 
a simple template fitting code that can be used in con- 
junction with two sets of basis SEDs, namely the observed 
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IColeman et alj (llOSd ) templates (CWW hereaft er) or the 
synthetically generated iBruzual fc CharlotI (|2003h templates 
(BC hereafter). HyperZ takes as its inputs the photometric 
catalogue of galaxies with magnitudes and errors on magni- 
tudes through the different filters specified in the filter set, 
as well as a list of spectral templates to be used in the 
minimisation. Various different reddening laws can also be 
implemented in order to account for the effect of interstellar 
dust on the shape of the SED. The damping of the Lyg for- 
est inc reasing with redshift is modelled according to lMadaul 
We have experimented with a variety of different 
basis template sets including the four CWW templates and 
interpolations between them as well as the 8 synthetic BC 
templates. We find the BC templates to produce consid- 
erably better photo-z's than the CWW and interpolated 
CWW template sets. In order to demonstrate the effects 
of using two different template sets to calculate photometric 
redshifts with the same code, we present results obtained 
using both the four CWW templates roughly correspond- 
ing to types E,Sbc,Scd and Im and the eight BC templates 
roughly corresponding to types Single Burst, E, SO, Sa, Sb, 
Sc, Sd and Im. 

We have also considered the photo-z outputs with no 
correction f or galactic reddening and with a Calzetti red- 
dening law jCalzetti et al.l Il994 ) applied to the templates 
for all the template sets considered. In all cases we find that 
including an extinction correction slightly worsens the pho- 
tometric redshift estimate. Our final HyperZ outputs there- 
fore make no correction for the galactic extinction. 

We used magnitudes in all five SDSS optical bands even 
though the photometric uncertainties in the ii-band are large 
and therefore would contribute to a larger scatter in the 
photo-z estimate. We have checked that removing the it- 
band data does in fact worsen the photo-z estimate. 

We note here that other template-based methods 
are also available for photo-z estimation such as ImpZ 

i private communication : M. Ro wan- Robinson) , k-co rrect 
Blanton fc Roweisll2007l). EAZY (jSrammer et alj2008l ) and 
the LRT Libraries ( Assef et al.ll2008l) . These have not been 
presented in this comparison. 

3.1.3 ANNz 

When a representative training set is available training 
methods become a viable option to use instead of template- 
fitting methods. The basic principle of training methods is 
the derivation of a parameterisation of redshift through the 
magnitudes of the galaxies in a training set. This parameter- 
isation is then applied to galaxies for which no spectroscopy 
is available, yielding an estimate of the photometric redshift. 
One of the training methods used here is Artificial Neural 
Networks (|Collister fc Lahavl [2003 ) . Neural networks have 
been used for estimation of photo-z in data (|Collister et al.l 
l2007l ) as well as forecasts of photometric redsh ifts for future 
data jBaneri i et al.ll200^ : lAbdalla et al.ll2008l ). An artificial 
neural network is made up of several layers, each consist- 
ing of a number of nodes. The first layer receives the galaxy 
magnitudes as inputs, while the last layer outputs the es- 
timated photometric redshift. The layers in between could 
consist of any number of nodes each. The nodes are inter- 
connected so that a node in a given layer is connected to 
all nodes in the adjacent layers, every connection carrying a 



weight Wij , where i and j describe the two nodes. Each node 
i is assigned a value Ui and an activation function gi{ui) 



I 



1 -f exp{~Ui) 



(2) 



is evaluated. 

The value of a subsequent node j is then calculated as 
the summation of the weighted values of the activation func- 
tions of all nodes i pointing to it: 



(3) 



When a network is trained the weights of all node con- 
nections are determined by minimising a cost function E 
evaluated on the training set of galaxies where 



^spec,k 



(4) 



and photometric input of irik for galaxy k from the train- 
ing set is Zphot{w,mk.), and the spectroscopic redshift of the 
galaxy is Zspec,k. 

To avoid an over-fitting, every network is tested on a 
validation set of gala^xies, whose spectroscopic redshifts are 
also known. The network with lowest value of E as calculated 
on the validation set is selected and the p hotometric sample 
is run through it for redshift estimation (jCoUister fc Lahavl 
l2004h . 

The artificial neural networks used in ANNz can be de- 
scribed as follows: Ni„ : Ni : N2 : ... : Nout, where Nin 
and Nout are respectively the number of input and output 
parameters, while Ni is the number of nodes in the i*'' inter- 
mediate layer. In the case of photometric redshift estimation 
using MegaZ-LRG, a network of the form 5 :10:10:1 was used , 
this was found empi rically to be optimal (|Firth et al.ll2003l : 
ICoUister etHlbOOTh . 



3.1.i BPZ 

An extension of the above HyperZ likelihood (x^) ap- 
proac h is to incorp orate priors, with the Bayesian frame- 
work. [Benfte^ (|2000l ) formulated the problem as follows. The 
probability of a galaxy with colour C and magnitude m hav- 
ing a redshift z is 



p{z\C,m) 



p{z\m)p{C\z) 
PiC) 



(xp{z\m)p{C\z), 



(5) 



where the term p{C\z) is the conventional redshift likelihood 
employed e.g. by Hyper-z, and p(C) is just a normalisation. 
The new important ingredient is p{z\m) , which brings in 
the prior knowledge of the magnitude redshift distribution. 
With the aid of the extra information (prior), this approach 
is effective in avoiding catastrophic errors of placing a galaxy 
at an unrealistic redshift. 

BPZ can function in a Bayesian and Maximum Like- 
lihood (ML hereafter) module and therefore produces two 
outputs for the photometric redshift. The ML method sim- 
ply picks the highest maximum over all the likelihoods as 
its redshift estimate whereas the Bayesian method averages 
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Figure 2. Density plots of spectroscopic versus photometric redshift for each of the public photo-z codes described in S 13.11 The plots 
are colour-coded and the scale is exponential. A colour difference of one is equivalent to the density being decreased by a factor of e. The 
solid black lines show whore the spectroscopic redshift is equal to the photometric redshift. 
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Spectroscopic Redshift Spectroscopic Redslnift 

Figure 3. Icr scatter on the photometric redshift around the true spectroscopic redshift defined as per Eq|6]for each of the pubfic photo-z 
codes described in § 13.11 in the left-hand panel and cres as a function of the spectroscopic redshift for each of the public photo-z codes 
described in ij lS.ll in the right-hand panel. 
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Figure 4. Icr scatter around the mean photometric redshift plotted as a function of spectroscopic redshift according to Eq[8](left panel) 
and bias as a function of spectroscopic redshift (right panel). We can see that there is a similar trend for most codes but a difference is 
present. In these metrics it seems that the training code is better suited to the scatter but not for the bias. We can see from the next 2 
figures that the opposite is true. 
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Figure 5. Icr scatter around mean spectroscopic redshift as a function of photometric redshift according to Eq|9]in the left panel and 
bias as a function of photometric redshift in the right panel. We can clearly see the power of the training code looking at the bias graph, 
ANNz performs with virtually no bias. However this has a certain drawback, the scatter around the mean spectroscopic redshift is larger 
in certain areas as a function of photometric redshift. Clearly different methods produce different quality results depending on the figure 
of merit used. We note e.g. that ANNz has limited coverage in photo-z as the training set is confined to that redshift range. 
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over all the likelihoods after weighting them by their prior 
probabilities. 

BPZ also takes as its input a photometric catalogue 
with magnitudes in different filters and their corresponding 
errors. The BPZ templates include the four CWW templates 
as well as the spec tra of two star-bursting galaxies from 
I Kinney et al.l (|l996l ) . In this study we added nine interpola- 
tions between each of the four CWW templates to the BPZ 
template list to produce a more complete list of basis SEDs. 
This gives us a set of 38 basis templates for BPZ to use. We 
find that ~17 of these templates concentrated towards the 
early types are sufficient to produce the best photometric 
redshift estimate. Adding more templates does not improve 
the photo-z scatter. We also used two further points of in- 
terpolation between each of the templates in colour space 
as specified by the INTERP parameter in BPZ. A flat prior 
was used throughout the calculation resulting in very similar 
results from the Bayesian and ML runs. 

The BPZ output includes two photo-z estimates from 
the Bayesian and ML runs as well as a quantity called 
odds that is the amount of probability contained between 
—0.12(1 -|- z) and 0.12(1 + z) around the Bayesian photo-z 
estimate. In order to select galaxies which only have a single 
compact peak in their probability distribution, we need to 
consider those galaxies with odds > 0.95 at the very least 
and odds > 0.99 for a robust estimate (private communica- 
tion: N.Benitez). In the flrst case, we select out 4811 of the 
5482 galaxies and in the second case we are left with 3689 
of the 5482 galaxies. The Bayesian output from BPZ with 
galaxies with odds > 0.99 selected, gives us the best photo-z 
estimate and it is this result that we use as the BPZ output 
in the plots. 

3.1.5 Le PHARE - PHotometric Analysis for Redshift 
Estimations 

Le PHARE is very similar to HyperZ in that a set of tem- 
plate SEDs together with a filter set are used to determine 
a set of model magnitudes used in the photometric redshift 
calculation. These are then compared to the observed mag- 
nitudes using a minimisation in order to compute the 
redshift of an object. The Le PHARE package includes var- 
ious template sets used to construct the library of model 
magnitudes. These include the Coleman, Wu and Weedman 
and Kinney star-burst templates, an extended CWW tem- 
plate set with 72 interpolations between the standard CWW 
templates, 42 synthetic GISSEL templates as well as the ob- 
served templates of Poggianti. We have experimented with 
using these various template sets for photo-z estimation on 
our sample of 5482 2SLAQ objects and find the best pho- 
tometric redshifts to be obtained with the 8 Poggianti tem- 
plates corresponding to galaxy types Ell, SO, Sa, Sb, Sc, Sd, 
SB2 and SBl. The 42 GISSEL templates give slightly worse 
photo-z's than the Poggianti templates but the scatter on 
the photometric redshift when using the extended sample of 
72 interpolated CWW templates is ---^30% worse than that 
obtained using the 8 Poggianti templates. Therefore we can 
see that in a template-based method, we do not necessarily 
gain in redshift accuracy by adding more model SEDs to our 
library. 

Le PHARE also includes various prescriptions to correct 
for galactic extinction. We have tried running Le PHARE 



with different extinction laws assuming E(B-V)=0.034 and 
find that in all cases, the photo-z estimate is worse when 
we include the effects of galactic extinction. Our final Le 
PHARE output is therefore obtained using the 8 Poggianti 
templates and neglecting the effects of galactic extinction. 
We use 5-band ugriz photometry in the SDSS filters as 
this gives significantly more accurate photometric redshifts 
compared to if we remove the w-band photometry with the 
largest photometric errors. 



3.1.6 ZEBRA - Zurich Extragalactic Bayesian Redshift 
Analyzer 

The Zurich Extragal actic Bayesian Redshift Analyzer 
(|Feldmann et al.l |2006| ) is a more sophisticated Bayesian 
template-fitting photometric redshift code compared to its 
predecessor, BPZ. The basic principles of estimating red- 
shifts using templates and Bayesian priors remains as de- 
scribed in i]3. 1.41 but among the novel techniques employed 
within the ZEBRA package are the photometry check mode 
that checks and corrects the photometry in certain filters, 
a template optimisation mode to improve the standard set 
of templates in specified redshift bins using a training set of 
galaxies with spectroscopic redshifts and the ability to calcu- 
late a prior self-consistently from the photometric catalogue 
when ZEBRA is run in its Bayesian mode. 

We choose not to employ the photometry check mode 
within ZEBRA as we are fairly confident that we have reli- 
able photometry for our objects and have checked that ap- 
plying a catalogue correction does not improve the photo- 
metric redshifts. 

zebra's template set consists of the standard E, Sbc, 
Scd and Im galaxies as well as the SEDs of the two star- 
bursting galaxies SB2 and SB3. These are further interpo- 
lated in logarithmic space by ZEBRA during the photo-z 
estimation. We find that including the E, Sbc and Scd tem- 
plates produces better results than including all six tem- 
plates. Furthermore, we use ZEBRA's template optimiza- 
tion mode to construct improved templates from these three 
basis templates in two redshift bins - < z < 0.5 and 
0.5 < z < 1.0. We use a regularisatio n parameter oi p = 0.05 
and a pliantness parameter of cr = 2. lFeldmann et all (|2006D 
gives details of these parameters and how to optimise them 
so as to produce the most realistic templates. We do not 
include IGM absorption in our templates as we find that 
this produces better photometric redshifts. The optimisa- 
tion procedure produces 39 basis templates and we use these 
along with our original templates in the photometric redshift 
calculation. 

We find the Bayesian mode of ZEBRA to produce con- 
siderably better photometric redshifts than the Maximum 
Likelihood mode and we therefore consider only outputs 
from the Bayesian mode when calculating figures of merit. 
The Bayesian mode is run using four iterations to calculate 
the prior self-consistently from the photometric catalogue. 
Further iterations slightly worsen the photo-z estimate. We 
use a smoothing kernel to smooth the prior after every iter- 
ation. 
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4 RESULTS 

Here we summarise the results obtained when running our 
dilTerent public photo-z codes on the 2SLAQ sample of 5482 
LRGs. Note that each of the photo-z codes detailed in § 13.11 
have been run several times using different parameters in or- 
der to optimise them to produce the best photo-z estimate. 
The final output files from each code used in this analysis 
are ones that gave the best photo-z estimate, hence this is 
not only a code comparison it is a code plus library com- 
parison which is the final publicly available product to the 
non-expert on-line. We note here that this is not a compar- 
ison which is meant to contrast equal values. For instance 
it is already well accepted that training codes work much 
better within the redshift and spectral energy distribution 
range present in the training set but template methods are 
superior if there are objects in the survey outside this range. 
Also, codes such as BPZ provide an automatic selection of 
the objects with the best photo-z's via for example the odds 
parameter. The purpose here is to compare the full packages, 
including SEDs and features available from different codes. 
Furthermore, the chosen sample of Luminous Red Galaxies 
has a very narrow range of SEDs and this comparison there- 
fore does not highlight the strength of photometric redshift 
codes with a broad range of library templates that would be 
more suitable for other samples. 

In Figure [2] we plot density plots of the spectroscopic 
redshift versus photometric redshift for each of these differ- 
ent codes. We use a redshift resolution for all codes which 
ensures that the main uncertainty is related to the photo-z 
uncertainty and not to numerical effects. 

In order to evaluate the precision with which each of 
these different codes calculates the photometric redshift, we 
can look at the Icr scatter between the true (spectroscopic) 
redshift and the photometric redshift. This is defined as fol- 
lows: 



Cz = {{Zphot - Zapec)^) ^ (6) 

This quantity is plotted in the left-hand panel of Fig- 
ure |3] As expected, the empirical photo-z estimator, ANNz 
seems to work best at intermediate redshifts where there 
are a large number of representative training set galaxies. 
At high redshifts, HyperZ BC provides us with the best es- 
timates of the photometric redshift. At low redshifts, the 
SDSS code and Le PHARE template fitting codes perform 
the best. We note that none of these runs used the more 
standard CWW templates suggesting that these templates 
are not a good match to the LRGs that are being analysed 
in this study. 

As can be seen in Figure [5] however, there are many 
outliers present in our sample. Another useful quantity to 
consider is therefore ues which is the interval in which 68% 
of the galaxies have the smallest difference between their 
spectroscopic and photometric redshifts. This will give us 
some indication of the scatter in the photometric redshift 
estimate once the outliers have been removed and is plotted 
in the right-hand panel of Figure [S] 

Another important quantity used to quantify how good 
a photo-z estimate is, is the bias defined as; 



This quantity is plotted for each o f the different codes, 
in the right-hand panel of Figure |4] IPadmanabhan et al.l 
show that galaxies with a given photometric redshift 
often have a systematic bias on them and this bias can there- 
fore be added to those photo-z galaxies in order to correct 
for it. In order to get a feel for the error on the photometric 
redshift once this bias has been corrected for, we plot in the 
left-hand panel of Figure |4j the Icr scatter around the mean 
photometric redshift estimate in each bin, defined as follows: 

Oz2 = {(Zphot - Zphot) ) ^ (8) 

As can be seen, the scatter is now reduced for most 
of the codes as we are not accounting for any systematic 
shift that can be corrected for. The bias is largest at high 
redshifts for the SDSS and Le PHARE template fitting codes 
and these codes have the biggest improvement in the scatter 
at high redshifts when we take the moment around the mean 
rather than the true redshift. 

A more useful quantity in terms of future surveys is to 
plot the bias and scatter as a function of the photometric 
redshift as in Figure [5] The la scatter around the mean 
spectroscopic redshift estimate in each photo-z bin is defined 
as follows: 

/ - 2\ - 

C^zS — \(^Zspec Zspec) / (9) 

We can also compare the right-hand panels of Figure |4] 
and Figure [5] to each other. We can see that the bias fol- 
lows the same trend as a function of spectroscopic redshift 
for all the different photo-z codes and is fairly similar for 
all these codes. However, the bias as a function of the pho- 
tometric redshift is very different for the different photo-z 
codes and more indicative of how much the photo-z estimate 
has to be corrected for systematic errors. This bias is almost 
flat for the training method which has enough training set 
galaxies to effectively minimise the bias through the train- 
ing process. The integral under the curve is also small for 
ZEBRA as shown in Table [3] as the template optimisation 
technique here was able to remove the average bias for the 
entire redshift range. However, a remaining bias was found 
at high and low redshifts with the values of the ZEBRA 
configuration parameters used in this analysi^. 

As an alternative statistic, we present in Table[5] the 
fraction statistics for the methods presented in this section. 
The fractions fi are defined as the fraction of galaxies in 
certain regions of the photo-z/spec-z plane. If we divide this 
plane in rectangular regions, the fraction fO is the fraction 
of galaxies which is on the diagonal of this matrix. We could 
have subdivided the areas along the diagonals, this would 

^ It is possible that a better choice of template optimisation pa- 
rameters could be found resulting in a removal of the bias at low 
and high redshifts. However, as the philosophy of this paper is to 
perform a code comparison from the point of view of the photo-z 
user rather than the photo-z developer, and it is not obvious what 
this better choice of parameters would be, wc choose to leave our 
results as they are. 
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Method 


/o 


/I 


/2 


/3 


/4 


Method 




bias 


ANNz 


37% 


28% 


31% 


4% 


0.02% 


ANNz 


0.0575 


0.0014 


SDSS 


27% 


31% 


37% 


4.5% 


0.8% 


SDSS 


0.0808 


-0.0264 


HyperZCWW 


24% 


32% 


35% 


7% 


1.5% 


HyperZCWW 


0.0973 


-0.0076 


HyperZBC 


26% 


32% 


32% 


7% 


1.5% 


HyperZBC 


0.0862 


0.0160 


LePHARE 


26% 


34% 


34% 


4% 


0.5% 


LePHARE 


0.0718 


-0.0302 


ZEBRA 


26% 


34% 


34% 


5% 


0.8% 


ZEBRA 


0.0898 


0.0013 


BPZ 


24% 


32% 


36% 


6% 


1% 


BPZ 


0.0933 


0.0112 



Table 2. Fraction statistics for the different codes presented in 
this section. The Fractions fO to f4 are defined in the following 
way. The galaxies are divided into a matrix defined along the 
axes in the photo-z/spec-z plane. The fraction fO is the fraction 
of galaxies which is on the diagonal of this matrix. The fraction fl 
is the fraction of galaxies in the first off diagonals of the matrix 
and so on. The grid are defined with the following boundaries 
in redshift z = [0.2,0.4,0.5,0.6,0.7,0.9]. We can see from the 
table that different methods provide different number of outliers. 
Given the nature of the galaxies the outlier fraction is small in 
each method but still relatively different across methods. 

have been a suitable statistic as well. However there are rea- 
sons for which the statistic we quote is interesting. For in- 
stance if one is interested in cosmological probes where the 
galaxies are separated in photo-z bins then a subdivision 
along the photo-z axis is more natural. Given that these 
galaxies are relatively red with good photometric redshifts, 
we can see that the fraction of outliers is small for all esti- 
mators, however there is still a difference between different 
implementations of publicly available codes. 

We also present in Table |3] the integrated bias and scat- 
ter for all the codes and libraries we have used. We can see 
that the training code performs best which is to be expected 
with a complete training set and that the bias here is very 
small. However this statistic does not show the redshift de- 
pendence of the scatter or bias which may be of interest 
depending on the application. 



5 SYSTEMATIC CHECKS ON THE 
PHOTOMETRIC CATALOGUES 

The clustering of the SDSS LRG photometric sample has 
been analysed using the SPSS code for photometri c red - 
shifts described in Sec l3.1.l] bv IPadmanabhaiLQ t all ( 20061 ) 
and by ANNz described in Sec l3.1.3lBlake et all (|2007l ). For 
the rest of this section we choose to look more in detail at 
the effects that these two codes have on the end products of 
the analysis. We check for gradients in the photo-z distribu- 
tion across the sky. These gradients should be present if the 
training set for the neural networks is biased as a function 
of position in the sky. 

5.1 Checking for Gradients in Redshift Difference 
across the Sky 

It is difficult to trust two very different techniques such as 
template fitting methods and training methods to produce 
consistent results without making a comprehensive compar- 
ison of both methods on the same set of galaxies. Given that 
the training set from the training method is drawn from a 



Table 3. Average Icr scatter (Eq. |6} and bias (Eq. [TJl for the 
entire sample for different methods. This is yet another metric 
to use if the redshift dependence of the bias and scatter is not of 
interest. Not e that the defin i tion o f the Icr scatter here is different 
from that in lCoUister et al.l l|2007t ) Eq. 10 




^ANNz~^SDSS Temn 



Figure 6. Histograms of the difference between the photometric 
redshifts from the ANNz code and from the SDSS template fitting 
code. We can see that the curves are identical apart from the 
normalization which is due to the different number of galaxies 
in each bin. this shows that the extinction is not producing a 
significant bias in the photometric redshifts given that one has a 
training set which is limited in the area in the sky. 

small region of the sky with limited range of galactic ex- 
tinction one could assume a priori that this extra calibra- 
tion which is necessary might introduce biases as a function 
of sky position in applications of empirical methods. This 
would be dramatic in the case of, for instance cosmologi- 
cal studies where we are attempting to calculate variations 
across the sky to infer cosmological parameters. 

We have looked for gradients in the difference dz = 
zannz — zsDSS in three redshift shells, 0.4 < Zphot < 0.5, 
0.5 < Zphot < 0.6 and 0.6 < Zphot < 0.7. We have separated 
galaxies in each redshift bin according to the ANNz photo- 
metric redshift. If the separation were done with the SDSS 
photometric redshifts instead the result would not change a 
lot. The values of dz were taken as an average value in pixels 
produced with HEALPIX, hence are smoothed to produce 
the maps in Fig[71 No apparent gradients can be identified, 
in any of the redshift shells, which is an indicator for the 
consistency across the plane of the sky. 

The different colour coding in each of the redshift shells 
in Fig[7] are indicative of the bias between the two meth- 
ods which is of course still present, however taking that bias 
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Figure 7. The bias between the photo-z estimate from ANNz and the photo-z estimate from SDSS. This has been subdivided in bins 
according to the photo-z estimate from ANNz. The redshift bins are 0.4 to 0.5 (top), 0.5 to 0.6 (middle) and 0.6 to 0.7 (bottom). We 
can clearly see from the colour coding that there is a bias as a function of redshift. If the bias is disregarded and wc look at the variation 
across the sky there is no evidence that there is a gradient or that the photometric rcdshifts are different closer to the regions where the 
training set was drawn from. The random nature of the residual bias shows that the extrapolation in sky position and calibrations are 
done to a sufficient accuracy and that the photo-z are statistically reliable in the plane of the sky. 
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Figure 8. Histogram of the difTorenco between the photomotrie redshift estimation between all pairs of codes we have used in this 
analysis. 
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Table 4. Parameters included in the updated McgaZ-LRG DR6 photometric rcdshift catalogue. 



objID 

ra 

dec 

dered_u 

dered_g 

dered_r 

dered_i 

dered_z 

err_u 

err.g 

err_r 

err_i 

err_z 

deVMag.i 

z_annz 

z_annz_err 

delta_sg 

delta_err_sg 

z_sdss 

z_hzcww 

z_hzcww_chi 

z_hzcww_errl 

z_hzcww_errh 

z_hzbc 

zjizbc_chi 

z_hzbc_errl 

z_hzbc_errh 

z_bpz_bayes 

z_bpz_errl 

z_bpz_errh 

z_bpz_odds 

z_bpz_inl 

z_bpz_chi 

z_zebra 

z_zebra_errl 

z_zebra_errh 

z_lp 

z_lp_prob 



SDSS objID 

J2000 right ascension 

J2000 declination 



Dereddened model magnitudes 



Magnitude errors 



Dereddened de Vaucouleurs magnitude 
ANNz photometric redshift 
ANNz photometric redshift error 
ANNz galaxy probability 
ANNz galaxy probability error 
SDSS photometric redshift 
Hyper-z CWW photometric redshift 
Hyper-z CWW chi squared 

Hyper-z CWW photometric redshift 68% lower confidence limit 
Hyper-z CWW photometric redshift 68% higher confidence limit 
Hyper-z BC photometric redshift 
Hyper-z BC chi squared 

Hyper-z BC photometric redshift 68% lower confidence limit 

Hyper-z BC photometric redshift 68% higher confidence limit 

BPz bayesian photometric rcdshift 

BPz photometric rcdshift 90% lower confidence limit 

BPz photometric redshift 90% higher confidence limit 

BPz bayesian odds parameter 

BPz maximum likelihood photometric redshift 

BPz chi squared 

ZEBRA photometric redshift 

ZEBRA photometric redshift 68% lower confidence limit 
ZEBRA photometric rcdshift 68% higher confidence limit 
Le PHARE photometric redshift 

Le PHARE percentage PDF between dz = z^,^gt it 0.1(1 -I- zj^st) 



aside, there seems to be no correlation between the usual re- 
gions of high extinction in the SDSS regions and the scatter 
of biases as a function of sky position produced here. Given 
that the template set does not know about the training set 
which belongs to a selective region of the sky our conclu- 
sion is that the fact that the training set is restricted to a 
small region of the sky does not include significant biases 
as a function of sky position and therefore is not an ex- 
tra source of systematic biases. This could be taken further, 
for instance by calculating spherical harmonics of the map 
above and comparing with theoretical predictions in order 
to estimate the actual lower bound of a potential systematic 
effect for a given probe but since this would involve a more 
specific cosmological approach we argue that this is beyond 
the photometric redshift comparison which is the aim of this 
paper. 

The main reason why there should be a systematic bias 
as a function of sky position is extinction. We have computed 
the difference between the two photo-z estimates for dif- 
ferent regions of galactic extinction and plotted histograms 
for these quantities in FiglHl Apart from the normalisation 
which encoded the fact that there are a different number 
of galaxies in these bins the curves are virtually identical 



with same bias and scatter, this also shows that there is 
no evidence of significant differences. This can only get bet- 
ter with future Planck data. So we are confident that this 
is not a systematic effect that will hinder future or current 
photometric redshift analysis. 

5.2 A Photo-z comparison of the codes 

We also present here a comparison of how the photometric 
redshifts of each code compare to each other. We present 
in Fig[8] histograms of the difference between photo-z for 
each pair of codes that we have used in the analysis of our 
updated MegaZ-LRG catalogue. We can see the differences 
between codes is apparent in some plots. 

For instance comparing ANNz to other codes there seem 
to be some outliers at a redshift difference of 0.1 compared to 
Hyperz CWW, BPZ and Hyperz BC. Similarly other pairs of 
codes produce outliers which indicates that this is not only 
a difference between template codes and training codes. We 
also note that for instance comparing ZEBRA with codes 
such as HyperZ CWW or SDSS code there is a good agree- 
ment on the scatter but there is a small bias between the 
codes which may suggest that the templates used might not 
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have been optimal in some codes. The ImpZ code (private 
communication:M. Rowan-Robinson) was also tested on the 
sample of 5482 2SLAQ LRGs and produced consistent re- 
sults with other template-based methods. However, as the 
code in its current form is not yet publicly available, we do 
not present these results here or extend the analysis to the 
MegaZ-LRG DR6 catalogue described in the next section. 

We emphasise here that there are many differences even 
though all the photo-z estimates are of relatively good qual- 
ity. There is therefore a need to deconstruct the effects of 
the algorithm and the template libraries in order for us to 
understand these differences and have even more reliable 
photo-z's in the future. 



6 AN EXTENSION TO MEGAZ-LRG: 
CATALOGUES WITH DIFFERENT 
PHOTO-Z ESTIMATORS FOR SDSS DR-6 

We have extended the photometric sample from SDSS DR4 
to SDSS DR6 imaging catalogue using the same criteria de- 
vised for the 2SLAQ LRG catalogue. This extended MegaZ- 
LRG catalogue contains 1543596 objects over more than 
8000 square degrees of the sky. As in previous studies LRGs 
are expected to be about 95% of the sample and M type 
stars are expected to be 5% of the sample. We have produced 
photometric redshift results for 7 different photometric red- 
shift estimators and provided the error estimators associated 
with each method. We also provide trained empirical val- 
ues to perform star/galaxy se paration based on a set of 15 
photometric parameters as in (|Collister et al.ll2007l ). All the 
parameters included in the revised catalogue are described 
in Table|4l The data can be found in the following websit^f]. 



7 CONCLUSIONS 

We have presented an updated version of the MegaZ-LRG 
catalogue. This catalogue contains about 1.5 million objects 
with accurate photometric redshifts which can be used for 
a range of science applications. The catalogue is available 
on-line and contains SDSS ID information so all SDSS data 
can be retrieved for each object as well as the photometric 
redshifts from each of the six public codes. 

We have run several comparisons of code and template 
libraries on the 2SLAQ LRG sample. We conclude that there 
are differences in the codes and stress that a more thorough 
comparison is needed where the effects of the codes and 
template libraries are disentangled. This will allow us to 
pinpoint where the discrepancies are arising. An approach 
based on first principles such as that presented in iBudavaril 
(|2008l ) is also timely. We have used several figures of merit 
to assess which code -I- template library performed best for 
this set of galaxies. We conclude that different codes perform 
with different strengths depending on the figure of merit 
used. We outline more specifically the findings below: 

• As expected, the availability of a complete training set 
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means the training method, ANNz performs best in the in- 
termediate redshift bins where there are plenty of spectro- 
scopic redshifts. 

• Le PHARE performs very well particularly in the lower 
redshift bins suggesting the Poggianti templates may be a 
better fit to LRGs at those redshifts compared to other tem- 
plates used in this comparison. 

• HyperZ run with Bruzual & Chariot templates gives 
better results than using the same with CWW templates 
once again highlighting the importance of template choice. 

• The SDSS template code gives very good results com- 
pared to other codes at the highest photo-z bins despite hav- 
ing only one evolving template for the LRGs. Given the nar- 
row range in SEDs of our sample of Luminous Red Galaxies, 
the strengths of template-based codes with extensive tem- 
plate libraries are not adequately highlighted by this com- 
parison. 

• ZEBRA shows a small average bias indicating the im- 
portance of the template optimisation technique in removing 
biases. 

As expected the training code performs best where the 
training set is large and complete and the template sets over- 
take the training code if the training set starts to become 
sparse. The importance of template choice is highlighted by 
the fact that most figures of merit show codes used in con- 
junction with the CWW templates to perform worse than 
those using other training or synthetic templates. This sug- 
gests that the CWW templates are not a very good match 
to the SEDs of these LRGs. 

There is a discrepancy between the scatters found for 
these codes ranging from 0.057 to 0.097. Both values are 
considered good results for photo-z estimates as one would 
expect from LRGs but there is a clear difference between 
the different code and template combinations that are run. 
Given that these differences will also depend on galaxy type 
and training set size, it is imperative that we carry out a 
more thorough comparison where the effects of codes and 
templates are deconstructed, in order to understand what 
factors affect the photo-z accuracy. We caution the reader 
that the results presented here are specific to a sub-sample of 
galaxies, namely LRGs, which have a narrow range in SEDs 
and a complete and representative spectroscopic training set 
available. The conclusions presented here could and proba- 
bly would change if the comparison were made with different 
galaxies or a different training set size. 

We have also produced a set of tests to assess whether 
the fact that the training sets are from a restricted area in 
the sky affects the photometric redshifts significantly. We 
conclude that there is little or no difference between the 
results from template methods and training set methods 
across the sky and that the difference found is not likely 
to be a source of the training set being restricted in area. 
This is promising for future wide-field photometric redshift 
surveys such as the Dark Energy Survey, PanStarrs, Euclid 
and JDEM. 
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